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The human ability to use symbols has yet to be replicated in machines. Bridging the gap requires 
considering how symbol meaning is established: if it is symbol users who agree-upon symbol meaning, 
then symbol-use comprises behaviours that navigate agreements about meaning. We leverage this 
insight to articulate graded symbolic behaviours, including constructing new symbols, altering prior 
ones, and introspecting about meaning and reasoning processes. We then evaluate contemporary AI 
methods along each criterion. Ultimately, we argue that fluent symbol-use in machines will emerge 
when learning-based agents immerse in human socio-cultural interactions demanding coordination 
around perspectives and meaning. By collecting these scenarios at scale, and including interactive 
human feedback, researchers can optimize towards symbolic fluency using contemporary algorithms 
for moulding behaviour. 


1. Symbols and Intelligence 


The ability to proficiently use symbols makes it possible to reason about the laws of nature, fly to the 
moon, write poetry, and evoke thoughts and feelings in the minds of others. Artificial intelligence 
(AI) pioneers Newell and Simon claimed that “[s]ymbols lie at the root of intelligent action” and 
should therefore be a central component in the design of artificial intelligence [1]. Their hypothesis 
drove a decades-long program of Good Old-Fashioned AI (GOFAD) research, which attempted to create 
intelligent machines by applying the syntactic mechanisms developed in computer science, logic, 
mathematics, linguistics, and psychology. 


Much contemporary artificial intelligence (AI) research strays from GOFAI methods and instead 
leverages learning-based artificial neural networks [2]. Neural networks have achieved substantial 
recent success in domains like language [3, 4] and mathematics [5], which were historically thought 
to require classical symbolic approaches. Despite these successes, some apparent weaknesses of 
current connectionist-based approaches [6, 7, 8] have lead to calls for a return of classical symbolic 
methods, possibly in the form of hybrid, neuro-symbolic models [9, 10, 11, 12, 13]. 


We suggest a different way forward that leverages a perspective on symbols originating in philoso- 
phy and semiotics [14], rather than classical logic and computer science [cf. 15]. This perspective 
emphasizes the triadic relation of meaning: Symbols are (1) entities that (2) mean something (3) 
to someone [16]. An interpreter (the aforementioned someone) exhibits an ability to come to an 
agreement—with interlocutors, or simply internally—about the arbitrary link between a substrate 
and what it denotes. 


The subjectivity of symbol meaning motivates viewing symbolic fluency through a distinctly be- 
havioural lens. This perspective is reminiscent of Wittgenstein’s proposal that we ought to understand 
language by examining how words are used; i.e., the associated behaviours (17; cf. 18, 19). Symbol 
use constitutes a subclass of behaviours that leverage the knowledge that symbol meaning is purely 
conventional [20], and that symbol meaning can, and should, be invented and continuously adapted. 


Consequently, instead of focusing dichotomously on whether a system (human, animal, or Al) 
engages with symbols, we focus on characterizing how it engages with symbols; that is, how it exhibits 
behaviours that implicate meaning-by-convention. This focus offers a set of behavioural criteria that 
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outline the varieties and gradations of symbolic behaviour. These criteria are measurable, which is 
useful both for assessing current AI, and as a direct target for optimization. 


The origin and development of symbolic behaviour in humans suggests a way to make progress 
towards developing AI that engages with symbols as humans do. Just as human symbolic fluency 
is built upon an ability to coordinate and communicate in social settings [21, 22], we propose 
subjecting learning-based machines to similar socio-cultural pressures, for example by learning from 
large datasets of human behaviours and by leveraging online interaction with humans and other 
agents [23]. Ultimately, it is through the combination of rich, challenging, diverse experiences of 
human-like socio-cultural interactions and powerful learning-based algorithms that we will develop 
machines that proficiently use symbols. 


2. Symbols Are Subjective 


Newell and Simon define symbols as a set of interrelated “physical patterns” that could “designate 
any expression whatsoever” [1]. But this definition—and how it is used in practice—is controversial. 
For example, Touretzky and Pomerleau [24] expressed frustration that some researchers render the 
concept of symbol operationally vacuous by claiming that anything that designates or denotes is a 
symbol. 


The philosopher Charles Sanders Peirce, many decades earlier, provided a complementary view. 
Peirce first outlined three categories of relation to establish meaning: icons, indices, and symbols [21, 
14]. Unlike icons (which make reference by way of similarity) or indices (which leverage some 
temporal or physical connection to the things to which they refer), symbols depend solely upon an 
“agreed upon link” to establish meaning. Thus, meaning does not depend on the actual physical or 
temporal characteristics of the medium. It is only by agreement that a flag comes to symbolize one 
country rather than another, and there is nothing inherent in the properties of cotton or ink that 
reveals the distinction. 


Newell, Simon, and Peirce therefore agree that a symbol consists of some substrate—a scratch on 
a page, an air vibration, or an electrical pulse—that is imbued with arbitrary meaning, but Peirce 
emphasizes the subjectivity of this convention. A substrate is only a symbol with respect to some 
interpreter(s), making symbols an irreducibly triadic phenomenon whose meanings are neither 
intrinsic to their substrate, nor objective [16]. 


Because symbols are inherently subjective constructs, their use and meaning is entangled with 
the state of their interpreters. As an interpreter’s background, capabilities, and biases vary, so too 
will their subjective treatment of symbols. This makes it problematic to draw binary distinctions 
between using and not using symbols. Indeed, as we will explore, symbol use manifests differently 
over the course of development, as broader cognitive systems and knowledge structures change [cf. 
25]. In the next section, we therefore outline graded criteria for assessing various aspects of symbolic 
behaviour—engagement with meaning-by-convention. 


3. Symbolic Behaviour 


An interpreter’s symbolic behaviour—their use of symbols—roughly comprises a set of interrelated 
traits that reveal how they participate in an infrastructure of meaning-by-convention. In particular, 
human symbolic behaviour is receptive, constructive, embedded, malleable, meaningful, and graded. In 
this section we elaborate on each dimension, and evaluate the progress of contemporary AI. 


Receptive. Symbolic behaviour includes the ability to appreciate existing conventions, and to 
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receive new ones. For example, humans can learn a new word from a definition or example. But 
many animals and models can learn such associations to some degree. So, while being receptive 
is necessary for participation in a symbolic framework, it is not sufficient for human-level symbolic 
fluency. 


Receptiveness is the predominant trait of current AI. For example, language models learn to 
imitate existing conventions [3], and multi-modal models learn about conventions of denotation, 
so that they can e.g. produce an image from a description [26]. Instruction-following agents map 
between symbolic inputs and goals [27]. Proponents of neuro-symbolic models often emphasize these 
models’ ability to rapidly learn a new concept, from a definition or a few examples [e.g. 28, 10, 29]. 
Deep learning models can also be receptive to established convention with only a single exposure (or a 
few exposures) to a new symbol definition [30, 3]. When transformers are trained to construct images 
from captions, these models can receive a composition of known concepts and generate plausible 
renderings of novel ones, such as a “daikon radish in a tutu walking a dog” [26]. Evidently, the ability 
to engage with established convention is broadly represented in AI research. 


Constructive. A second trait of symbolic behaviour is the ability to form new conventions; because 
meaning is conventional, it can be imposed arbitrarily on top of any substrate. Such an ability can 
be used to increase the efficiency of communication or reasoning (e.g. by creating a new term for a 
recurring situation) and for creating new systems of knowledge. Consider, for example, the scientific 
understanding which followed the introduction of the concept of the “genes” to describe units of 
heritable variability. 


Evidence for this behavioural trait is less common in current AI research. Few-shot learning [31] 
explores a model’s ability to rapidly assign meaning to a new substrate, but the meaning is pre- 
determined by humans. Some neuro-symbolic or classical models can construct abstractions that 
capture some useful structure [e.g. 32, 33], but so far these abilities have only been demonstrated 
within very simplified and algebraically-structured domains. Deep learning models can also create 
abstractions, e.g. summaries of documents [34] or code [35]. Some ability to construct novel concepts 
emerges in large language models (see Box 1), but only when prompted. Stronger evidence for this 
behavioural trait might look like the following: (i) A model is asked a question, and when explaining 
its answer it invents a new symbol to clarify its reasoning process or communication. (ii) Agents 
cooperating on a team invent terminology so that they can communicate without exposing their 
strategy to their opponents. (iii) A model constructs a novel mathematical concept in order to prove 
something new (cf. Box 2.) 


Embedded. Symbols conform and contribute to the broader symbolic system in which they’re 
situated, and symbolic behaviour reflects this understanding [21]. Because the meaning of a symbol 
is determined partly through its interactions with the entire symbol system,introducing new symbols 
can radically alter the interpretation of other symbols. For example, the discovery of category theory 
as a unifying perspective on many mathematical research areas fundamentally changed the questions 
researchers ask [36]. Humans use embodied understanding, e.g. gestures, to help them grasp abstract 
concepts [37, 38], and domain knowledge aids logical reasoning in Wason Selection Task analogues 
that frame a logical problem in terms of a social situation [39, 40, 41]. The whole structure of 
knowledge in which symbols are embedded can, and usually does, affect symbol-use. 


Current AI models demonstrate some embedded behaviour. Neural networks in particular have 
strong representational and functional biases toward such behaviour. For example, continuous vector 
representations are meaningful with respect to the magnitudes and angles of other vectors, and such 
vector relations are directly shaped by gradient-based learning to be useful for a downstream task, such 
as classification [42]. For this reason, deep learning models can suffer from poor generalization [6] 
without sufficient inductive biases, or data from which to learn the interrelationships between 
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symbols. However, performance generally improves when symbols are embedded within a richer, 
more embodied contexts [43, 44], or even using auxiliary data to learn symbol-symbol relations, as 
is accomplished with pre-trained word embeddings [45]. These continuous vectors capture complex 
relationships like analogies [46], providing concrete evidence for the utility of this embedded property 
(though there are also drawbacks, such as capturing biases in language use [47]). 


Malleable. Fluent symbol users understand that the conventionality of meaning allows for change. 
Meaning can be altered by context, by the creation of other symbols (see “embedded”, above) or 
concepts, or by intentionally redefining the symbol. 


The collaborative and pragmatic nature of language exemplifies malleability [48, 49]. Language 
interpretation depends heavily on the shared situation and the mutual understanding between the 
speaker and listener, and so understanding human communication requires an ability to continu- 
ally form pragmatic conventions [48, 50]. Beyond a single situation, malleability is often used in 
constructing new symbols too as language evolves—for example, in the way that “computer mouse” 
constructs a new concept by building on and transforming the existing word “mouse.” 


Furthermore, permanently altering the meaning of an existing symbol can be useful. Deep insight 
can require epistemic humility—recognizing that symbol meaning could be otherwise, or should be 
otherwise. For example, extending the concept of numbers to include complex numbers allowed 
humans to mathematically describe new phenomena. To reason flexibly, it is necessary to reshape 
extant meanings that are misaligned with the world or other symbols. But to do this, one must first 
appreciate that meaning can be reshaped because it is established by convention. 


It’s not obvious whether any AI system exhibits malleable understanding of symbols as humans 
do. Language models can discriminate different word senses [51] and exhibit some aspects of 
pragmatic understanding [52]. These models demonstrate impressive abilities to learn the many 
subtle constraints that determine language meaning in context, and will likely improve when they 
are augmented with more human-like faculties and grounded experience [44]. However, it’s less 
clear whether these models exhibit behaviour that demonstrates that they can, or should, decide to 
actively shift their understanding of an already known symbol. Any learning-based models will surely 
alter its understanding of a symbol as it experiences new data. However, this type of malleability is 
passive, as it depends on researchers to provide the data from which new meaning could be derived. 
Humans, on the contrary, are malleable with purpose, whether that purpose is to permit more fluid 
communication, or to come to a deeper understanding of some phenomenon. Such intentional action 
motivates techniques that exploit the situated, goal-directed aspects cognition, such as reinforcement 
learning [53, 54, 55]. 


Meaningful. Symbolic behavior should demonstrate use and communication of the meaning 
behind the symbols and the reasoning processes that employ them. While Newell [56] advocated 
for purely syntactic manipulations, we emphasize that syntactic operations do not consider the 
steps where meaning is extracted from, and later imbued in, arbitrary physical tokens. Focusing 
on syntactic manipulations assumes that such manipulations would be useful to a system that has 
yet to demonstrate an understanding of symbols as symbols. Instead, even in logical domains like 
mathematics, meaning dominates human reasoning (Box 2). Analogously, Marder [57] describes 
theory in biology as “disciplined dreaming” wherein we face the “challenge of creatively marrying the 
rules of mathematics and physics with what is known of fundamental biological principles.” 


However, meaning is more than a heuristic for syntactic reasoning. Models must understand 
their reasoning processes themselves as meaningful. Harnad [58] alluded to this fact in his focus on 
grounding tokens, noting that “the syntactic manipulations both actual and possible and the rules 
[must be] ‘semantically interpretable.” The principles by which the model manipulates its symbols 
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must themselves be understood symbolically by the system. 


Meaningful symbolic behaviour therefore involves both using semantics to reason more effectively 
(e.g. finding a proof of a theorem efficiently rather than blindly searching) and communicating how 
the semantics licenses the reasoning process (e.g. why a particular proof technique is useful, or which 
steps are critical). Many recent models have used deep learning to partially address the first point. 
For example, AlphaZero [59] uses learned move value estimates as strong heuristics for reducing its 
search space. However, because AlphaZero’s reasoning process (Monte-Carlo Tree Search) is hand 
engineered and strictly rule-based, it cannot achieve the second goal. Analogous limitations apply to 
neuro-symbolic models that use deep learning within a more programmatic reasoning process [e.g. 
32, 60]. This motivates future research toward systems that can understand their reasoning processes 
as meaningful, and use that understanding to refine and communicate their reasoning. 


Meaningful reasoning is central to critical issues of safety and ethics. For example, we echo 
others who highlighted the challenges of creating rule-based machine ethics [e.g. 61, 62]. Without 
understanding the meaning behind the rules, such systems would only follow the letter of the law, not 
the spirit. Consider a rule like “don’t discriminate on the basis of race.” As US history unfortunately 
illustrates, it’s easy to find a proxy variable for race, like neighborhood, and have essentially the same 
discriminatory effect [63]. We need a system that behaves in accordance with the meaning behind its 
principles—a system with judgement [64]. We suggest that this goal will require pursuing AI that 
can exhibit symbolic behavior within a holistic, meaningful framework of ethics. 


Graded. Finally, symbol use is not binary [c.f. 25, 65]. Each of the behavioural criteria is best 
described as a spectrum of competencies. For example, young children might be receptive to simple 
language conventions, but unable to construct useful conventions themselves. Similarly, few humans 
may be able to identify certain useful alterations to meaning, because most may lack the relevant 
knowledge and experience (consider how scientific insights, such as a heliocentric solar system, are 
often initially conceived by only one or a few individuals). 


Symbolic thinking is therefore not an independent, isolated function, but is rather an interde- 
pendent component of our broader cognition. And since other aspects of our cognition—attention, 
memory, perception—can be graded, so too can our ability to think symbolically. As it regards AI, 
acknowledging that the ability to use symbols is graded entails acknowledging that symbol use is not 
synonymous with the existence a particular computation (e.g. syntactic control over entities), or the 
use of a particular type of representation (e.g. discrete tokens), see Box 3. We argue instead for char- 
acterizing behaviourally how a system expresses engagement with symbols. If certain computations or 
representations are essential, this should be demonstrated through the behavioural competence of a 
system that includes them when performing rich tasks. Thus, in the next section we draw inspiration 
from the development of symbolic behaviour in humans to suggest a path towards achieving more 
human-like symbolic behaviour in artificial intelligence. 
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Box 1 | Symbolic behaviour in large language models 


Language was traditionally regarded as a quintessentially GOFAI domain. However, neural networks 
without symbolic components have advanced rapidly in natural language (NLP). Transformers [66]— 
models that use attention to process a sequence—are achieving particularly remarkable successes 
[e.g. 67, 3]. When trained to simply predict the next word (or word-piece) on internet text, these 
models demonstrate flexibility: they perform passably on tasks like translation, difficult Winograd 
schemas [68], and programming problems [69]. They do so even without explicit training on those 
tasks, from a task description or a few examples [67, 3]. When fully trained on translation [70], 
programming [69], or mathematics (see 2), transformers substantially outperform existing symbolic 
or hybrid models. Furthermore, the representations in transformer models are highly predictive of 
human brain activity during language processing [71], suggesting that they capture some aspects of 
human language processing. These models also achieve some symbolic behaviour, which we assess 
below. 

Receptive: Large models can correctly conjugate and use a novel verb after receiving a definition [3]. 
Malleable: Language models exhibit some pragmatics [e.g. 51]. However, they do not exhibit 
intentionally malleable behaviour, such as changing meanings to propose a better definition. 
Constructive: We contributed a test of constructing a concept uniting disparate concepts (e.g. the- 
oretical particle physics, tennis rackets, and sewing all involve strings) to the BIG-Benchmark [72, 
“novel_concepts” task], and language models exhibit some success. But, there is little evidence that 
these models could devise useful constructions in the pursuit of some goal. 

Embedded & meaningful: Despite arguments that models cannot understand meaning from lan- 
guage alone [73], recent research shows that transformers do build implicit models of situations, 
entities, and relations [74]. Thus, it is possible to learn aspects of meaning from language alone [cf. 
18, 19], though these models would still benefit from embedding in more human-like capacities and 
grounding (44; cf. 75). 

However, there is little evidence that language models understand their own reasoning processes as 
meaningful or intentional. Transformers can sometimes manipulate their own outputs, e.g. reducing 
bias after reprocessing [76] or correcting a program they produced with a human explanation of 
their mistakes [69]. But, to exhibit these behaviours, the system designers need to carefully engineer 
the way the outputs are processed and reprocessed by the system; the model lacks the understanding 
and control necessary to manipulate its own reasoning. 

Graded: In summary, these models exhibit some gradations of symbolic behaviour, but not full 
symbolic fluency. These models lack human-like embedding, and are mostly passive recipients, rather 
than constructing or changing symbols in pursuit of some goal. These limitations would be addressed 
by the interactive, socio-cultural immersion we suggest below. 
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Box 2 | Syntax & meaning in mathematics 


Newell & Simon motivated the physical symbol system hypothesis by this claim: “Logic, and by 
incorporation all of mathematics, was a game played with meaningless tokens according to certain 
purely syntactic rules. All meaning had been purged.” [1]. In the reasoning systems they created, 
symbols thus became synonymous with discrete entities, symbolic processing entailed manipulating 
with formal algebras, and the properties of our formal symbolic systems were inherited by GOFAI. 
But, is this how humans perform logical reasoning? Fields medalist Paul Cohen responded directly: “I 
never was able to successfully analyze proofs as a combinatorial ‘game’ played with symbols on paper,” 
instead, to reason productively “one must essentially forget that all proofs are eventually transcribed 
in this formal language” [77]. Mac Lane [78] explains why: “strict formalism can’t explain which of 
many formulas matter [...] the choice of form is determined by ideas and experience.” The meaning 
of symbols and their relationships gives mathematicians intuitions for why a theorem might be true, 
and thereby ideas of how to prove it (as do examples; 79). Furthermore, a major goal of theories and 
proofs is to convey understanding. Thus, mathematicians are often unsatisfied with a formal proof of 
a theorem that does not clearly communicate understanding of the underlying ideas [36, 80]. The 
meaning behind symbols is the essence of mathematics, not a burden to be discarded. 

McClelland et al. [65] draw on similar arguments to suggest that neural network models are uniquely 
well-posed to capture the meaning-laden processes of human mathematical reasoning. Indeed, deep 
learning models are increasingly succeeding at difficult mathematical reasoning tasks like identifying 
the theorem needed to complete a proof [81], or solving difficult symbolic integration [5], without 
GOFAI or neuro-symbolic methods. Lample and Charton [5] even show that transformers in some 
cases outperform classical symbol systems like Mathematica that were specifically designed for these 
tasks, though there are certainly caveats to that particular comparison [82]. Deep learning models 
will likely improve further when they can learn from a more human-like curriculum—even simple 
reasoning curricula can improve these models performance [83]. Similarly, while it is difficult for 
transformers to learn to perform division from problems and answers alone, even with millions of 
examples [84], a pretrained language model endowed with a virtual notepad and trained on only 200 
detailed demonstrations of long division can achieve 80% generalization to new problems [85]. Even 
beyond the curriculum, humans build from physical experience to abstractions [86, 75], e.g. gestures 
help us learn mathematics [38, 37], and deep learning might likewise benefit from such grounding. 
Furthermore, current models do not generally reason about their own reasoning processes, or interpret 
them as meaningful. We suggest that achieving this will be important for achieving fully human-like 
mathematical abilities. 


4. How Can Symbolic Behaviour Come About? 


Some traditional theories suggest that symbolic competency could be enabled by unique events in 
evolution that equip humans with “innate” mental organs. [87, 88]. However, other theories posit 
that the human ability for symbolic thought may have instead resulted from a gradual internalization 
of evolved behaviours involved in socio-cultural interaction and communication [16, 21, 89, 90]. That 
is, the cognitive requirements for outwardly establishing and appreciating meaning-by-convention 
among a set of interlocutors consequently permit internal symbolic thought, setting up a virtuous, 
incremental cycle of co-development [21, 16]. 


This process may have occurred on an evolutionary time-scale in humans, but even within single 
lifetimes humans gain critical experience learning the consequences of meaning-by-convention. For 
example, children must learn that words will be known by speakers of the same language (but 
not others), and if they are bilingual, they also learn how to not mix languages [91]. They also 
learn to consider speaker knowledge and intent [92], reinterpret or ignore noun-object bindings if a 
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speaker is unreliable [93], and infer social norms from others’ behaviour [94]. Tomasello [22] argues 
persuasively that the ability to reconcile different perspectives is prerequisite to the ability to manage 
convention, and hence, to coordinate on symbol meaning and use. 


Analogously, fluent symbol-use in machines will emerge when situated, learning-based agents are 
immersed in scenarios that demand an active participation with meaning-by-convention. Compared 
to pure multi-agent settings, immersion in human socio-cultural situations forces agents to cope with, 
and learn from, the diverse symbolic behaviours that humans already exhibit. Practically, human 
experiences can be compiled into enormous datasets from which agents can imitate rich symbolic 
behaviours, and human-agent interactions can be gathered in real-time so that humans can provide 
behavioural feedback signals that cannot be programmed a priori. In both cases, these experiences 
can be obtained at scale, which allows for the use of supervised, self-supervised, and reinforcement 
learning to learn human-like symbolic behaviours. 


Some recent work has begun to pursue directly optimizing natural human-agent interactions 
at scale. In [23], humans and agents take control of avatars in a virtual 3D environment called a 
“Playroom”, comprising objects such as furniture and toys. Setter agents (or humans) pose a task for 
Solver agents (or humans) using natural language, such as “pick up the blue duck and bring it to 
the bedroom”. Using imitation learning techniques, the authors were able to train artificial agents to 
display impressive aspects of symbolic behaviour, such as embeddedness (consider how the meaning 
of “duck” co-depends on other symbols, perceptions, motor behaviours, and goals) and malleability 
(to handle coreferents like “it” and “that”). We see this as a promising approach to placing agents in 
situations requiring conventional, perspectival, and pragmatic interaction with humans. Critically, 
the authors collected a vast quantity of data—the agents learned from more than 600,000 episodes 
of human-human interaction in a complex environment. Collecting rich compilations of symbolic 
behaviours at scale allows for direct optimization towards these competencies. However, further 
work is needed to create experiences that develop particular aspects of symbolic behaviour, such as 
constructive and meaningful behaviours. 


This argument aligns with a broad perspective in contemporary AI that robust cognitive faculties 
emerge because we optimize learning-based systems for desired behaviours directly, rather than 
designing internal mechanisms that may ultimately prove limiting [95, 55]. It has roots in cognitive 
science [e.g. 25, 96], and is now more broadly supported by findings of recent research in AI: 
researchers often progressively remove designed knowledge in favor of the power of general learning 
systems, for example from AlphaGo [97], to AlphaZero [59], and MuZero [98]. Moreover, even 
if some knowledge or ability is “innate” in humans, it must have emerged through evolutionary 
pressures acting on experiences, which can be captured by meta-learning or other learning-based 
algorithms in artificial settings [99]. Indeed, evolution-like competition and selection of models across 
a few generations has supported many recent succeses of RL [e.g. 100, 101] and language [e.g. 102]. 


The idea that aspects of intelligence are emergent is not new [e.g. 103, 104], but recent find- 
ings in AI reinforce that scale (both of models and data) may be critical to achieving human-like 
behaviour—the lesson that “more is different” [103] is true in artificial intelligence as well. For 
example, larger models learn more than smaller models from the same amount of data [105], and in 
some cases exhibit qualitatively different behaviors, such as acquiring new capacities for reasoning [3, 
Fig. 3.10] or generalization [4, Fig. 6B]. Scale is not the only factor, however; interactive feedback 
can be more valuable than orders of magnitude of scale [54]. Furthermore, generalization can be 
qualitatively different when experience is more interactive or embodied [43], and training tasks are 
more diverse [101]. Immersion in human culture comprises the best of all these worlds because it 
can dramatically increase the scale, richness, diversity, and interactivity of our agents’ experiences. 
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Box 3 | Reconciling our view with other perspectives 


The ongoing debate about symbols in AI mostly revolves around the properties of certain ways 
of computing [9, 10, 11, 12, 13]. By contrast, our behavioural perspective does not prescribe the 
functions or mechanisms required for symbolic fluency. Advocates for GOFAI-like mechanisms 
often also cite behavioural evidence, particularly humans’ relatively systematic compositional 
generalization, which is often assumed to rely on GOFAI-like compositional mechanisms [e.g. 106]. 
Our views differ in several crucial ways (see also 107, 43). 


First, under our interpretation symbol-use does not entail perfect systematicity. We acknowledge 
the graded, flawed nature of human behaviour—even adult humans are often far from perfectly 
systematic, achieving compositional generalization performance of only 70-90% even within 
relatively simple domains [108]. Furthermore, we emphasize that systematic behaviour does not 
comprise a more pure mode of engaging with symbols. In some cases, such approaches could even 
be detrimental; natural language, for example, is probably not strictly compositional [109, 110]. 
Enforcing strict algebraic notions of compositionality a priori will make it impossible to capture the 
full structure of such domains. Indeed, the graded compositionality afforded by neural networks 
may play a role in their superior performance on language understanding compared to systems 
with stricter assumptions [44]. Our perspective embraces these lessons, and does not conflate 
symbol-use with the potential properties of an underlying symbolic framework. This view also 
implies that mechanisms that guarantee systematicity may be neither necessary nor sufficient for 
symbolic behaviour. Rather, systematicity may be a graded competency afforded by environmental 
and educational factors [25, 43, 44, 111]. 


Second, symbols are often defined as discrete, localist representations [12, e.g.]. That view fits with 
computers’ binary representations, and thus allows leveraging computer algorithms in Al. Humans 
can interpret non-discrete entities as symbolic—for example, continuous variations in intonation, 
volume, and pitch of a single word can convey vastly different emotions, sarcasm, and so on. The 
important distinction is that a particular entity need not be discrete to be a member of a discrete set; 
the representation of a word can be continuous but treated as a discrete symbol, just as a continuous 
vector can be a member of a discrete set of basis vectors. If we are to appreciate symbolic thought in 
its most potent, general form, then we need to embrace the idea that symbols come in many shapes 
and forms. According to the definition presented here, any substrate can be symbolic to a fluent 
symbol user. The triadic relation of the substrate, the denotation, and the interpreter are all that is 
needed, not any particular properties of the substrate. 


5. Conclusion 


Classical perspectives on symbols in AI have mostly overlooked the fact that symbols are fundamentally 
subjective—they depend on an interpreter (or some interpreters) to create a convention of meaning. 
Thus, human-like symbolic fluency is not guaranteed simply because a system is equipped with 
classical “symbolic” machinery. Instead, symbolic fluency should be evaluated through behaviours, 
whether these behaviours involve interactions with interlocutors or simply demonstrate improved 
internal reasoning.This can be measured by inspecting a set of graded traits, such as receptiveness 
to new convention, the ability to construct new conventions, and demonstrated understanding of 
the meaning behind syntactic maneuvers. Because optimizing directly for behaviour is increasingly 
feasible, we argue that the key to developing machines with human-like symbolic fluency is to optimize 
learning-based systems for these symbolic behaviours directly by placing artificial agents in situations 
that require their active use. Human socio-cultural situations are perhaps best suited to fulfill this 
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requirement, as they demand the complex coordination of perspectives to agree on arbitrarily-imposed 
meaning. They can also be collected at scale in conjunction with human feedback, and hence allow 
the use of powerful contemporary AI tools that mould behaviour. 
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